The short answer: no, but yes too (better read the long answer!)
From 'inside' the privately derived class (ie: in the body of members or friends of the privately derived class), the relationship to the base class is known, and the upward conversion from PrivatelyDer* to Base* (or PrivatelyDer& to Base&) is safe and doesn't need a cast.
From 'outside' the privately derived class, the relationship to 'Base' is a 'private' decision of 'PrivatelyDer', so the conversion requires a cast. Clients should not exercise this cast, since private derivation is a private implementation decision of the privately derived class, and the coercion will fail after the privately derived class privately chooses to change this private implementation decision.
Bottom line: only a class and its friends have the right to convert a ptr to a derived class into a ptr to its private base class. They don't need a cast, since the relationship with the base class is accessible to them. No one else can convert such ptrs without pointer-casts, so no one else should.